{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fintech AI Agent\n",
    "\n",
    "![Fintech-AI-Agent.excalidraw.png](https://github.com/lancedb/vectordb-recipes/blob/main/examples/fintech-ai-agent/Fintech%20AI%20Agent.excalidraw.png?raw=true)\n",
    "\n",
    "These days, there are tons of frameworks out there to help you whip up AI agents fast. Honestly, it feels like a new “low-code” tool shows up every other day, promising to make building agents a breeze. They’re all over the place—simple drag-and-drop setups or text boxes where you plug in a few prompts and some data, and just like that, your agent’s ready to go.\n",
    "\n",
    "This time, we will go all in to create a fintech AI agent with no framework, everything is in raw python. This notebook carries an imporant appication of the fintech industry where people face enormus amount of time in processsing loan queries and insurance claims."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "executionInfo": {
     "elapsed": 3166,
     "status": "ok",
     "timestamp": 1740578903198,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "vdDufJritdEO"
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "!pip install pandas joblib pyarrow sentence-transformers lancedb mistralai huggingface_hub"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setting up the data needs and libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 2856,
     "status": "ok",
     "timestamp": 1740578929434,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "9isZFPr2ualv",
    "outputId": "9c8f44ce-ef6f-45b4-acba-601a5922f1a5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2025-02-26 14:08:46--  https://drive.google.com/uc?export=download&id=1huZAkbQzTwiELbKbNpghbC2hEBLOqW9V\n",
      "Resolving drive.google.com (drive.google.com)... 173.194.217.139, 173.194.217.113, 173.194.217.102, ...\n",
      "Connecting to drive.google.com (drive.google.com)|173.194.217.139|:443... connected.\n",
      "HTTP request sent, awaiting response... 303 See Other\n",
      "Location: https://drive.usercontent.google.com/download?id=1huZAkbQzTwiELbKbNpghbC2hEBLOqW9V&export=download [following]\n",
      "--2025-02-26 14:08:46--  https://drive.usercontent.google.com/download?id=1huZAkbQzTwiELbKbNpghbC2hEBLOqW9V&export=download\n",
      "Resolving drive.usercontent.google.com (drive.usercontent.google.com)... 173.194.217.132, 2607:f8b0:400c:c13::84\n",
      "Connecting to drive.usercontent.google.com (drive.usercontent.google.com)|173.194.217.132|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 1804682 (1.7M) [application/octet-stream]\n",
      "Saving to: ‘credit_risk_dataset.csv’\n",
      "\n",
      "credit_risk_dataset 100%[===================>]   1.72M  --.-KB/s    in 0.02s   \n",
      "\n",
      "2025-02-26 14:08:49 (102 MB/s) - ‘credit_risk_dataset.csv’ saved [1804682/1804682]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "!wget --no-check-certificate 'https://drive.google.com/uc?export=download&id=1huZAkbQzTwiELbKbNpghbC2hEBLOqW9V' -O credit_risk_dataset.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "F5R6vGuqqIaw"
   },
   "source": [
    "### LLM and Embedding Model Customization\n",
    "\n",
    "For the language model operations, I’ve implemented Mistral as the foundation, delivering reliable and efficient performance tailored to various tasks. However, the setup is highly flexible—you can seamlessly switch to any other provider or model that better suits your specific needs, ensuring adaptability for your unique use case. The same level of customization applies to the embedding model, allowing you to choose an alternative that aligns perfectly with your project requirements. If you decide to stick with this configuration, just remember to include your `Hugging Face token` and `Mistral API key` to authenticate and unlock the full potential of the system without any hiccups.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "executionInfo": {
     "elapsed": 645,
     "status": "ok",
     "timestamp": 1740578966172,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "D09u5aVquZ1K"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import joblib\n",
    "import pyarrow as pa\n",
    "from sentence_transformers import SentenceTransformer\n",
    "import lancedb\n",
    "from mistralai import Mistral\n",
    "import os\n",
    "from abc import ABC, abstractmethod\n",
    "from typing import Any\n",
    "\n",
    "\n",
    "# Abstract Tool Class\n",
    "class Tool(ABC):\n",
    "    @abstractmethod\n",
    "    def name(self) -> str:\n",
    "        pass\n",
    "\n",
    "    @abstractmethod\n",
    "    def description(self) -> str:\n",
    "        pass\n",
    "\n",
    "    @abstractmethod\n",
    "    def use(self, *args, **kwargs) -> Any:\n",
    "        pass\n",
    "\n",
    "\n",
    "# Initialize Mistral\n",
    "api_key = os.environ.get(\"MISTRAL_API_KEY\", \"kER2XXXXXXXXXXX\")\n",
    "if not api_key:\n",
    "    raise ValueError(\"Please set the MISTRAL_API_KEY environment variable.\")\n",
    "model = \"mistral-large-latest\"\n",
    "client = Mistral(api_key=api_key)\n",
    "hf_token = \"hf_XXXXXXXXXXXXX\"\n",
    "\n",
    "# Initialize Embedder\n",
    "embedder = SentenceTransformer(\"all-MiniLM-L6-v2\", token=hf_token)\n",
    "\n",
    "# Connect to LanceDB\n",
    "db = lancedb.connect(\"./lancedb_data\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predictive ML Modelling\n",
    "\n",
    "I’m using the [Credit Risk Dataset](https://www.kaggle.com/datasets/laotse/credit-risk-dataset) as a demo for loans. In reality, companies could use something like a CIBIL score to see if someone’s eligible. But for this POC, I grabbed that dataset, trained a random forest model, and used it for predictive analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 226
    },
    "executionInfo": {
     "elapsed": 145,
     "status": "ok",
     "timestamp": 1740578968682,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "HQz0VxC_wNI0",
    "outputId": "d47239c9-24c7-4e96-c47f-96713d664995"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "summary": "{\n  \"name\": \"loan_data\",\n  \"rows\": 32581,\n  \"fields\": [\n    {\n      \"column\": \"person_age\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 6,\n        \"min\": 20,\n        \"max\": 144,\n        \"num_unique_values\": 58,\n        \"samples\": [\n          22,\n          26,\n          65\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"person_income\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 61983,\n        \"min\": 4000,\n        \"max\": 6000000,\n        \"num_unique_values\": 4295,\n        \"samples\": [\n          20800,\n          54417,\n          144000\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"person_home_ownership\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 4,\n        \"samples\": [\n          \"OWN\",\n          \"OTHER\",\n          \"RENT\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"person_emp_length\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 4.142630148887972,\n        \"min\": 0.0,\n        \"max\": 123.0,\n        \"num_unique_values\": 36,\n        \"samples\": [\n          30.0,\n          18.0,\n          26.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"loan_intent\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 6,\n        \"samples\": [\n          \"PERSONAL\",\n          \"EDUCATION\",\n          \"DEBTCONSOLIDATION\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"loan_grade\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 7,\n        \"samples\": [\n          \"D\",\n          \"B\",\n          \"F\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"loan_amnt\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 6322,\n        \"min\": 500,\n        \"max\": 35000,\n        \"num_unique_values\": 753,\n        \"samples\": [\n          13125,\n          3675,\n          13800\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"loan_int_rate\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3.2404594649559195,\n        \"min\": 5.42,\n        \"max\": 23.22,\n        \"num_unique_values\": 348,\n        \"samples\": [\n          10.28,\n          13.35,\n          18.07\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"loan_status\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"loan_percent_income\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.10678175634492885,\n        \"min\": 0.0,\n        \"max\": 0.83,\n        \"num_unique_values\": 77,\n        \"samples\": [\n          0.55,\n          0.34\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"cb_person_default_on_file\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"N\",\n          \"Y\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"cb_person_cred_hist_length\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 4,\n        \"min\": 2,\n        \"max\": 30,\n        \"num_unique_values\": 29,\n        \"samples\": [\n          24,\n          25\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
       "type": "dataframe",
       "variable_name": "loan_data"
      },
      "text/html": [
       "\n",
       "  <div id=\"df-a1a972ea-0ac3-4983-b508-63fd4ac213d4\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>person_age</th>\n",
       "      <th>person_income</th>\n",
       "      <th>person_home_ownership</th>\n",
       "      <th>person_emp_length</th>\n",
       "      <th>loan_intent</th>\n",
       "      <th>loan_grade</th>\n",
       "      <th>loan_amnt</th>\n",
       "      <th>loan_int_rate</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>loan_percent_income</th>\n",
       "      <th>cb_person_default_on_file</th>\n",
       "      <th>cb_person_cred_hist_length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>22</td>\n",
       "      <td>59000</td>\n",
       "      <td>RENT</td>\n",
       "      <td>123.0</td>\n",
       "      <td>PERSONAL</td>\n",
       "      <td>D</td>\n",
       "      <td>35000</td>\n",
       "      <td>16.02</td>\n",
       "      <td>1</td>\n",
       "      <td>0.59</td>\n",
       "      <td>Y</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>21</td>\n",
       "      <td>9600</td>\n",
       "      <td>OWN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>EDUCATION</td>\n",
       "      <td>B</td>\n",
       "      <td>1000</td>\n",
       "      <td>11.14</td>\n",
       "      <td>0</td>\n",
       "      <td>0.10</td>\n",
       "      <td>N</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>25</td>\n",
       "      <td>9600</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>1.0</td>\n",
       "      <td>MEDICAL</td>\n",
       "      <td>C</td>\n",
       "      <td>5500</td>\n",
       "      <td>12.87</td>\n",
       "      <td>1</td>\n",
       "      <td>0.57</td>\n",
       "      <td>N</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>23</td>\n",
       "      <td>65500</td>\n",
       "      <td>RENT</td>\n",
       "      <td>4.0</td>\n",
       "      <td>MEDICAL</td>\n",
       "      <td>C</td>\n",
       "      <td>35000</td>\n",
       "      <td>15.23</td>\n",
       "      <td>1</td>\n",
       "      <td>0.53</td>\n",
       "      <td>N</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>24</td>\n",
       "      <td>54400</td>\n",
       "      <td>RENT</td>\n",
       "      <td>8.0</td>\n",
       "      <td>MEDICAL</td>\n",
       "      <td>C</td>\n",
       "      <td>35000</td>\n",
       "      <td>14.27</td>\n",
       "      <td>1</td>\n",
       "      <td>0.55</td>\n",
       "      <td>Y</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a1a972ea-0ac3-4983-b508-63fd4ac213d4')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-a1a972ea-0ac3-4983-b508-63fd4ac213d4 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-a1a972ea-0ac3-4983-b508-63fd4ac213d4');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-be423d39-9e58-4ff7-8bba-83ed3ad98869\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-be423d39-9e58-4ff7-8bba-83ed3ad98869')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-be423d39-9e58-4ff7-8bba-83ed3ad98869 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "   person_age  person_income person_home_ownership  person_emp_length  \\\n",
       "0          22          59000                  RENT              123.0   \n",
       "1          21           9600                   OWN                5.0   \n",
       "2          25           9600              MORTGAGE                1.0   \n",
       "3          23          65500                  RENT                4.0   \n",
       "4          24          54400                  RENT                8.0   \n",
       "\n",
       "  loan_intent loan_grade  loan_amnt  loan_int_rate  loan_status  \\\n",
       "0    PERSONAL          D      35000          16.02            1   \n",
       "1   EDUCATION          B       1000          11.14            0   \n",
       "2     MEDICAL          C       5500          12.87            1   \n",
       "3     MEDICAL          C      35000          15.23            1   \n",
       "4     MEDICAL          C      35000          14.27            1   \n",
       "\n",
       "   loan_percent_income cb_person_default_on_file  cb_person_cred_hist_length  \n",
       "0                 0.59                         Y                           3  \n",
       "1                 0.10                         N                           2  \n",
       "2                 0.57                         N                           3  \n",
       "3                 0.53                         N                           2  \n",
       "4                 0.55                         Y                           4  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data = pd.read_csv(\"./credit_risk_dataset.csv\")\n",
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6S4CVnoAxH7a"
   },
   "source": [
    "### ML Modelling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "executionInfo": {
     "elapsed": 22,
     "status": "ok",
     "timestamp": 1740578972455,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "3lN676UZxGfP"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.compose import ColumnTransformer\n",
    "\n",
    "# Select relevant features and target\n",
    "features = [\"person_age\", \"person_income\", \"loan_amnt\", \"loan_intent\"]\n",
    "target = \"loan_status\"\n",
    "\n",
    "# Handle missing values by dropping rows with NaN in these columns\n",
    "loan_data = loan_data.dropna(subset=features + [target])\n",
    "\n",
    "# Define preprocessing for categorical feature 'loan_intent'\n",
    "preprocessor = ColumnTransformer(\n",
    "    transformers=[(\"cat\", OneHotEncoder(handle_unknown=\"ignore\"), [\"loan_intent\"])],\n",
    "    remainder=\"passthrough\",  # Keep numerical features as is\n",
    ")\n",
    "\n",
    "# Split data into features (X) and target (y)\n",
    "X = loan_data[features]\n",
    "y = loan_data[target]\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.2, random_state=42\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 4068,
     "status": "ok",
     "timestamp": 1740578978082,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "8eiT4PKHxMA4",
    "outputId": "52e987f4-e584-430a-ef64-2018129a51e2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model accuracy: 0.84\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.pipeline import Pipeline\n",
    "\n",
    "# Create and train the model pipeline\n",
    "model_pipeline = Pipeline(\n",
    "    steps=[\n",
    "        (\"preprocessor\", preprocessor),\n",
    "        (\"classifier\", RandomForestClassifier(random_state=42)),\n",
    "    ]\n",
    ")\n",
    "model_pipeline.fit(X_train, y_train)\n",
    "\n",
    "# Evaluate the model\n",
    "accuracy = model_pipeline.score(X_test, y_test)\n",
    "print(f\"Model accuracy: {accuracy:.2f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 404,
     "status": "ok",
     "timestamp": 1740578978498,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "69kkPLFuxPAh",
    "outputId": "e2dac9f0-95ce-46bc-bb3f-ff4e0220f89b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['loan_approval_model.pkl']"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import joblib\n",
    "\n",
    "# Save the trained model\n",
    "joblib.dump(model_pipeline, \"loan_approval_model.pkl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RJBxfbbBxQos"
   },
   "source": [
    "### Setting up the Kernel Agent\n",
    "\n",
    "The Kernel Agent serves as the intelligent core of the system, designed to comprehend user queries and determine the appropriate actions to direct them to the correct agent—be it the loan agent or the insurance agent—ensuring a smooth and efficient process. It leverages key details provided by the user, such as parameters, along with the detected intent behind the query, to make informed decisions and route requests accurately to the most relevant agent for resolution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "executionInfo": {
     "elapsed": 11,
     "status": "ok",
     "timestamp": 1740578978511,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "ABYskyOTxRu1"
   },
   "outputs": [],
   "source": [
    "class KernelAgent:\n",
    "    def __init__(self, loan_agent, insurance_agent):\n",
    "        self.loan_agent = loan_agent\n",
    "        self.insurance_agent = insurance_agent\n",
    "        self.client = client\n",
    "        self.model = model\n",
    "\n",
    "    def classify_query(self, query):\n",
    "        prompt = f\"\"\"\n",
    "        Given the query: '{query}', classify it as 'loan' or 'insurance'.\n",
    "        Respond with only 'loan' or 'insurance'.\n",
    "        If unsure, respond with 'unknown'.\n",
    "        \"\"\"\n",
    "        response = self.client.chat.complete(\n",
    "            model=self.model,\n",
    "            messages=[{\"role\": \"user\", \"content\": prompt}],\n",
    "            temperature=0.7,\n",
    "            max_tokens=10,\n",
    "        )\n",
    "        result = response.choices[0].message.content.strip().lower()\n",
    "        # print(f\"Query classified as: {result}\")\n",
    "        return result\n",
    "\n",
    "    def process_query(self, query, params=None):\n",
    "        query_type = self.classify_query(query)\n",
    "        if query_type == \"loan\":\n",
    "            if not params:\n",
    "                return (\n",
    "                    \"Error: Loan query requires parameters (age, income, loan_amount).\"\n",
    "                )\n",
    "            return self.loan_agent.process(query, params)\n",
    "        elif query_type == \"insurance\":\n",
    "            return self.insurance_agent.process(query)\n",
    "        else:\n",
    "            return \"Error: Unable to classify query as 'loan' or 'insurance'.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "executionInfo": {
     "elapsed": 31,
     "status": "ok",
     "timestamp": 1740578978561,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "j5KFz2T6xSzY"
   },
   "outputs": [],
   "source": [
    "class PredictiveMLTool(Tool):\n",
    "    def __init__(self):\n",
    "        self.model = joblib.load(\"loan_approval_model.pkl\")\n",
    "\n",
    "    def name(self):\n",
    "        return \"Predictive ML Tool\"\n",
    "\n",
    "    def description(self):\n",
    "        return \"Predicts loan eligibility using a trained ML model.\"\n",
    "\n",
    "    def use(self, intent, age, income, loan_amount):\n",
    "        input_data = pd.DataFrame(\n",
    "            [[age, income, loan_amount, intent]],\n",
    "            columns=[\"person_age\", \"person_income\", \"loan_amnt\", \"loan_intent\"],\n",
    "        )\n",
    "        prob_approved = self.model.predict_proba(input_data)[0][0]\n",
    "        print(f\"Probability for loan approval: {prob_approved:.3f}\")\n",
    "        return \"Eligible\" if prob_approved > 0.5 else \"Not Eligible\"\n",
    "\n",
    "\n",
    "class LoanAgent:\n",
    "    def __init__(self, client, model):\n",
    "        self.tools = [PredictiveMLTool()]\n",
    "        self.client = client\n",
    "        self.model = model\n",
    "\n",
    "    def extract_intent(self, query):\n",
    "        valid_intents = [\n",
    "            \"DEBTCONSOLIDATION\",\n",
    "            \"EDUCATION\",\n",
    "            \"HOMEIMPROVEMENT\",\n",
    "            \"MEDICAL\",\n",
    "            \"PERSONAL\",\n",
    "            \"VENTURE\",\n",
    "        ]\n",
    "        prompt = f\"\"\"\n",
    "        Given the query: '{query}', classify the intent into one of:\n",
    "        {', '.join(valid_intents)}.\n",
    "        Respond with only the intent in uppercase (e.g., 'HOMEIMPROVEMENT').\n",
    "        If unsure, respond with 'PERSONAL'.\n",
    "        \"\"\"\n",
    "        response = self.client.chat.complete(\n",
    "            model=self.model,\n",
    "            messages=[{\"role\": \"user\", \"content\": prompt}],\n",
    "            temperature=0.7,\n",
    "            max_tokens=20,\n",
    "        )\n",
    "        intent = response.choices[0].message.content.strip().upper()\n",
    "        return intent if intent in valid_intents else \"PERSONAL\"\n",
    "\n",
    "    def process(self, query, params):\n",
    "        intent = self.extract_intent(query)\n",
    "        age = params.get(\"age\")\n",
    "        income = params.get(\"income\")\n",
    "        loan_amount = params.get(\"loan_amount\")\n",
    "        return self.tools[0].use(intent, age, income, loan_amount)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1475,
     "status": "ok",
     "timestamp": 1740580069937,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "hZjZmdRPxUpc",
    "outputId": "56a8bf2f-1296-4f36-b414-2a91140dac5c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Probability for loan approval: 0.960\n",
      "Query: 'I need a loan for home renovation' | Params: {'age': 30, 'income': 50000, 'loan_amount': 10000} -> Result: Eligible\n",
      "\n",
      "Probability for loan approval: 0.440\n",
      "Query: 'I need a loan for medical expenses' | Params: {'age': 25, 'income': 20000, 'loan_amount': 25000} -> Result: Not Eligible\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Test cases for LoanAgent\n",
    "test_cases = [\n",
    "    {\n",
    "        \"query\": \"I need a loan for home renovation\",\n",
    "        \"params\": {\"age\": 30, \"income\": 50000, \"loan_amount\": 10000},\n",
    "    },\n",
    "    {\n",
    "        \"query\": \"I need a loan for medical expenses\",\n",
    "        \"params\": {\"age\": 25, \"income\": 20000, \"loan_amount\": 25000},\n",
    "    },\n",
    "]\n",
    "\n",
    "# Initialize LoanAgent\n",
    "loan_agent = LoanAgent(client, model)\n",
    "\n",
    "# Run test cases\n",
    "for case in test_cases:\n",
    "    query = case[\"query\"]\n",
    "    params = case[\"params\"]\n",
    "    result = loan_agent.process(query, params)\n",
    "    print(f\"Query: '{query}' | Params: {params} -> Result: {result}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b1TO-m98xZAW"
   },
   "source": [
    "### Generating the Demo Insurance Claim dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before diving in, I want to walk you through my thought process for creating the Insurance Agent. Picture this: you walk into an insurance company, sit down with someone, and start explaining why you need to file a claim—why it’s legit and why you deserve it. That conversation is key because it’s the starting point for deciding whether the claim gets processed or not.\n",
    "\n",
    "I tried hunting down a dataset with those kinds of conversations and clear yes or no labels for claims, but I came up empty. So, to keep things moving, I built a simple synthetic dataset pipeline. It generates a fake dataset with text queries on one side and a target on the other—showing if the person filing the claim is eligible or not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 41800,
     "status": "ok",
     "timestamp": 1740579029018,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "1_Dwy08oxZK-",
    "outputId": "0e751228-1fe9-4a21-c1a0-64047b3401ec"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generating 10 Auto Insurance Claim Queries:\n",
      "--------------------------------------------------\n",
      "1. Query: Hi, I'm reaching out about an auto insurance claim I need to file. Last night, I was involved in a single-car accident in Massachusetts. I was driving home from work around 8 PM, and it was raining quite heavily. I hydroplaned and lost control of my vehicle, crashing into a guardrail. I was alone in the car, and thankfully, I only have some minor bruises and a slight concussion, but my car is pretty badly damaged. The front end is smashed, and the airbags deployed, so I'm sure the repairs will be costly.\n",
      "\n",
      "I have comprehensive coverage with a $500 deductible. I'm worried, though, because I had a glass of wine at a work event before driving home. I didn't think I was impaired, but I'm concerned that this could affect my claim. I know I should have called a cab, but I\n",
      "   Target: 0 (Not Approved)\n",
      "--------------------------------------------------\n",
      "2. Query: I was involved in a rear-end collision last week in California. I was stopped at a red light when another driver hit me from behind. The impact caused significant damage to the rear bumper and trunk of my car, and I also have some neck pain that I'm getting checked out by a doctor. The other driver admitted they were distracted by their phone and didn't see me stopped in time. I have their insurance information, but I'm worried because I had a lapse in my insurance coverage for a few days last month. I forgot to pay the bill on time, but I did reinstate it before the accident. Could this lapse affect my claim? I'm looking to get the damage to my car repaired and potential medical expenses covered. What are the next steps I should take, and is there any chance my claim could be denied due to the lapse in coverage?\n",
      "   Target: 0 (Not Approved)\n",
      "--------------------------------------------------\n",
      "3. Query: Hi, I'm calling about an auto insurance claim I need to file. I was involved in a car accident yesterday in California. I was driving on the highway when another driver merged into my lane without signaling and hit the front passenger side of my car. The airbags deployed, and my car is pretty badly damaged—the bumper is dented, the headlight is broken, and there are scratches all along the side. I took photos of the damage and have the other driver's insurance information. Fortunately, I only have some minor bruises, but I did go to the emergency room to get checked out, and they recommended I follow up with my primary care doctor. I'm looking to claim for the repairs to my vehicle and potential medical expenses, as I'm not sure how much the follow-up appointments will cost. Could you guide me through the process of filing this claim and let me know what other information you might need from me?\n",
      "   Target: 1 (Approved)\n",
      "--------------------------------------------------\n",
      "4. Query: Dear Insurance Agent,\n",
      "\n",
      "I hope this message finds you well. I am writing to inquire about filing an auto insurance claim following a recent accident. On October 15, 2023, at approximately 7:30 PM, I was involved in a collision in the state of California. The accident occurred on Interstate 5, near Exit 33.\n",
      "\n",
      "The incident was a rear-end collision. I was traveling at the speed limit when another vehicle hit me from behind. The cause of the accident appears to be the other driver's inattention; they were distracted by their phone. The damages to my vehicle include significant rear bumper damage, a broken taillight, and possible frame issues. Fortunately, I sustained only minor injuries—a few bruises and a sore neck. The other driver also suffered minor injuries.\n",
      "\n",
      "I would like to claim for the damages to my vehicle and any necessary medical expenses. My policy\n",
      "   Target: 0 (Not Approved)\n",
      "--------------------------------------------------\n",
      "5. Query: Hi, I hope you're doing well. I need to file an auto insurance claim and I'm looking for some guidance on the process. Last weekend, I was involved in a rear-end collision in California. I was stopped at a red light in downtown Los Angeles when the driver behind me didn't brake in time and hit my car from the back. The impact was quite significant; my car has considerable damage to the rear bumper and trunk, and the airbags were deployed. I also sustained some injuries—mainly whiplash and a few bruises. I went to the emergency room right after the accident and have the medical reports and bills. The other driver admitted fault and we exchanged insurance information. I have their details as well as the police report from the scene.\n",
      "\n",
      "My car is a 2018 Toyota Corolla, and I'm not sure if it's drivable in its current condition. I\n",
      "   Target: 1 (Approved)\n",
      "--------------------------------------------------\n",
      "6. Query: I was involved in a car accident last week in California and need to file a claim. I was driving my 2018 Honda Civic on the I-5 freeway when another driver merged into my lane without signaling and hit the passenger side of my car. The impact caused me to lose control and hit the guardrail. The other driver's insurance company has already accepted liability.\n",
      "\n",
      "My car has significant damage to the passenger side, including dents, scratches, and a broken side mirror. The front bumper was also damaged from the impact with the guardrail. I took my car to a local auto body shop, and they estimated the repairs to be around $4,500. I also had to rent a car while mine is in the shop, which is costing me $35 a day.\n",
      "\n",
      "I sustained some minor injuries—mainly bruises and whiplash. I visited the ER immediately\n",
      "   Target: 1 (Approved)\n",
      "--------------------------------------------------\n",
      "7. Query: Hi, I need to file an auto insurance claim. Last night, I was involved in a rear-end collision in California. I was driving home from work on the freeway when traffic suddenly slowed down. Unfortunately, the driver behind me didn't brake in time and hit the back of my car. The impact was significant; my car's rear bumper is severely damaged, and the trunk is crumpled. I also have some whiplash and bruising from the impact.\n",
      "\n",
      "I've been to the hospital and have the medical reports and bills ready. I also have photos of the damage to my vehicle and a copy of the police report. The other driver admitted fault and gave me their insurance information, but I'm worried because they mentioned they might have let their policy lapse. I need to claim for the damages to my car and the medical expenses. However, I'm concerned my claim might be affected because I had a glass of wine\n",
      "   Target: 0 (Not Approved)\n",
      "--------------------------------------------------\n",
      "8. Query: I was involved in a car accident last week in California, and I need to file a claim. I was driving my 2018 Honda Civic on the freeway when another driver merged into my lane without signaling and hit the side of my car. The accident caused significant damage to the driver's side door and rear panel, and I also have some minor injuries—bruises and a slight sprain in my neck. The other driver was cited for an unsafe lane change, but I'm worried because I was driving while slightly over the legal alcohol limit, which I know was a terrible mistake.\n",
      "\n",
      "I want to know if my comprehensive coverage will pay for the damages to my car and if my medical payments coverage will cover my hospital bills. I'm also concerned about whether my claim might be denied due to the fact that I was driving under the influence. Can you guide me through the process and let me know what to expect?\n",
      "   Target: 0 (Not Approved)\n",
      "--------------------------------------------------\n",
      "9. Query: Hi, I hope you're doing well. I need to file an auto insurance claim for an accident that occurred yesterday in California. I was driving my 2018 Honda Civic on the I-5 freeway when another driver merged into my lane without signaling and hit the rear passenger side of my car. The impact caused me to lose control and crash into the median. Fortunately, there were no injuries, but my car has significant damage. The rear bumper is severely dented, the passenger side door is scraped, and the rear taillight is broken. The other driver admitted fault, and we exchanged insurance information. I have his name, contact details, and insurance policy number. I also took photos of the scene and the damages to both vehicles.\n",
      "\n",
      "I need to know the next steps to file a claim for the damages to my car. Specifically, I want to understand:\n",
      "\n",
      "1. What documents or information do I need\n",
      "   Target: 1 (Approved)\n",
      "--------------------------------------------------\n",
      "10. Query: Hi, I'd like to inquire about filing an auto insurance claim. Last night, I was involved in a rear-end collision in California. I was driving home from work on the freeway when traffic suddenly slowed down. Unfortunately, the driver behind me didn't brake in time and hit my car from the rear. The impact deployed my airbags and caused significant damage to my bumper, trunk, and rear windshield. I also sustained some minor injuries—whiplash and bruises from the airbag. I went to the emergency room and was treated and released.\n",
      "\n",
      "My car is a 2018 Honda Civic, and I have comprehensive coverage with a $500 deductible. The other driver's insurance company has already accepted liability. However, I'm worried because I had a glass of wine at a work event before the accident. Although I don't feel I was impaired, I\n",
      "   Target: 0 (Not Approved)\n",
      "--------------------------------------------------\n",
      "Total queries generated: 10\n",
      "Dataset saved to 'auto_insurance_claims.csv'\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "import csv\n",
    "\n",
    "# Define base prompt for insurance claims\n",
    "base_prompt = \"\"\"\n",
    "Generate a detailed query for an auto insurance claim. Include specifics about the incident (e.g., cause of the accident, damages, injuries, type of accident, state where it occurred), the user's situation, and what they're claiming. The query should be realistic and reflect what a user might ask an insurance agent. Output only the query text, nothing else.\n",
    "\"\"\"\n",
    "\n",
    "\n",
    "def generate_query(denied=False):\n",
    "    prompt = base_prompt\n",
    "    if denied:\n",
    "        prompt += \" Include a reason the claim might be denied (e.g., drunk driving, uninsured vehicle).\"\n",
    "    response = client.chat.complete(\n",
    "        model=model,\n",
    "        messages=[{\"role\": \"user\", \"content\": prompt}],\n",
    "        temperature=0.7,\n",
    "        max_tokens=200,\n",
    "    )\n",
    "    return response.choices[0].message.content.strip()\n",
    "\n",
    "\n",
    "def assign_target(query):\n",
    "    rejection_keywords = [\"drunk\", \"influence\", \"racing\", \"uninsured\"]\n",
    "    return 0 if any(keyword in query.lower() for keyword in rejection_keywords) else 1\n",
    "\n",
    "\n",
    "# Generate 10 queries for simplicity (increase to 100 as needed)\n",
    "dataset = []\n",
    "print(\"Generating 10 Auto Insurance Claim Queries:\")\n",
    "print(\"-\" * 50)\n",
    "for i in range(10):\n",
    "    denied = random.random() < 0.4\n",
    "    query = generate_query(denied)\n",
    "    target = assign_target(query) if not denied else 0\n",
    "    dataset.append({\"query\": query, \"target\": target})\n",
    "    print(f\"{i+1}. Query: {query}\")\n",
    "    print(f\"   Target: {target} ({'Approved' if target == 1 else 'Not Approved'})\")\n",
    "    print(\"-\" * 50)\n",
    "\n",
    "# Save to CSV\n",
    "with open(\"auto_insurance_claims.csv\", \"w\", newline=\"\") as f:\n",
    "    writer = csv.DictWriter(f, fieldnames=[\"query\", \"target\"])\n",
    "    writer.writeheader()\n",
    "    writer.writerows(dataset)\n",
    "\n",
    "print(f\"Total queries generated: {len(dataset)}\")\n",
    "print(\"Dataset saved to 'auto_insurance_claims.csv'\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b5H2QIATlbCw"
   },
   "source": [
    "### Inserting Demo Insurance Queries into LanceDB\n",
    "\n",
    "The demo insurance queries are added to LanceDB to serve as a foundation for future operations, enabling semantic search capabilities across the existing query dataset. When a new query is introduced, it will be embedded once more, allowing the system to leverage hybrid search techniques to effectively explore and match it against the pre-existing database, ensuring accurate and efficient retrieval of relevant information.\n",
    "\n",
    "This can become a go-to for the fintech comapaines where they can plug in their own Insurance Conversations logs and do the semantic search on the new embeded queries to retrive the eligibility for the claim."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "executionInfo": {
     "elapsed": 1693,
     "status": "ok",
     "timestamp": 1740579030722,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "OsXGRCIzldf_"
   },
   "outputs": [],
   "source": [
    "# Load insurance dataset\n",
    "df_insurance = pd.read_csv(\"auto_insurance_claims.csv\")\n",
    "\n",
    "# Generate embeddings\n",
    "embeddings = embedder.encode(df_insurance[\"query\"].tolist())\n",
    "embeddings_list = [embedding.tolist() for embedding in embeddings]\n",
    "\n",
    "# Define schema and create table\n",
    "schema = pa.schema(\n",
    "    [\n",
    "        pa.field(\"embedding\", pa.list_(pa.float32(), list_size=384)),\n",
    "        pa.field(\"target\", pa.int32()),\n",
    "        pa.field(\"query\", pa.string()),\n",
    "    ]\n",
    ")\n",
    "table = db.create_table(\"insurance_queries\", schema=schema, mode=\"overwrite\")\n",
    "df_lance = pd.DataFrame(\n",
    "    {\n",
    "        \"embedding\": embeddings_list,\n",
    "        \"target\": df_insurance[\"target\"],\n",
    "        \"query\": df_insurance[\"query\"],\n",
    "    }\n",
    ")\n",
    "table.add(df_lance)\n",
    "\n",
    "\n",
    "# Define SemanticSearchTool\n",
    "class SemanticSearchTool(Tool):\n",
    "    def __init__(self, table):\n",
    "        self.table = table\n",
    "        self.embedder = embedder\n",
    "\n",
    "    def name(self):\n",
    "        return \"Semantic Search Tool\"\n",
    "\n",
    "    def description(self):\n",
    "        return \"Performs semantic search in LanceDB to assess claim approval based on similar past claims.\"\n",
    "\n",
    "    def use(self, query, k=5):\n",
    "        new_embedding = self.embedder.encode([query])[0].tolist()\n",
    "        results = self.table.search(new_embedding).limit(k).to_pandas()\n",
    "        approval_rate = results[\"target\"].mean()\n",
    "        similar_queries = results[\"query\"].tolist()\n",
    "        decision = \"Approved\" if approval_rate > 0.5 else \"Not Approved\"\n",
    "        print(f\"Approval rate among similar claims: {approval_rate*100:.1f}%\")\n",
    "        return {\"decision\": decision, \"similar_queries\": similar_queries}\n",
    "\n",
    "\n",
    "# Define InsuranceAgent\n",
    "class InsuranceAgent:\n",
    "    def __init__(self, table):\n",
    "        self.tools = [SemanticSearchTool(table)]\n",
    "\n",
    "    def process(self, query):\n",
    "        return self.tools[0].use(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Puttting it all together!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1589,
     "status": "ok",
     "timestamp": 1740580201752,
     "user": {
      "displayName": "Vipul Maheshwari",
      "userId": "13637702448104644568"
     },
     "user_tz": -330
    },
    "id": "1sbBf-nKlerx",
    "outputId": "dcde1ae3-feac-45b6-8bde-2daa82d8f725"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Testing Integrated System with Mistral:\n",
      "--------------------------------------------------\n",
      "Probability for loan approval: 0.960\n",
      "Query: 'I need a loan for home renovation'\n",
      "Params: {'age': 30, 'income': 50000, 'loan_amount': 10000}\n",
      "Result: Eligible\n",
      "--------------------------------------------------\n",
      "Approval rate among similar claims: 20.0%\n",
      "Query: 'I had a bad accident yesterday, but I think I was a bit drunk, need some insurance'\n",
      "Decision: Not Approved\n",
      "Similar Queries:\n",
      "- Hi, I'm reaching out about an auto insurance claim I need to file. Last night, I was involved in a single-car accident in Massachusetts. I was driving home from work around 8 PM, and it was raining quite heavily. I hydroplaned and lost control of my vehicle, crashing into a guardrail. I was alone in the car, and thankfully, I only have some minor bruises and a slight concussion, but my car is pretty badly damaged. The front end is smashed, and the airbags deployed, so I'm sure the repairs will be costly.\n",
      "\n",
      "I have comprehensive coverage with a $500 deductible. I'm worried, though, because I had a glass of wine at a work event before driving home. I didn't think I was impaired, but I'm concerned that this could affect my claim. I know I should have called a cab, but I\n",
      "- Hi, I'd like to inquire about filing an auto insurance claim. Last night, I was involved in a rear-end collision in California. I was driving home from work on the freeway when traffic suddenly slowed down. Unfortunately, the driver behind me didn't brake in time and hit my car from the rear. The impact deployed my airbags and caused significant damage to my bumper, trunk, and rear windshield. I also sustained some minor injuries—whiplash and bruises from the airbag. I went to the emergency room and was treated and released.\n",
      "\n",
      "My car is a 2018 Honda Civic, and I have comprehensive coverage with a $500 deductible. The other driver's insurance company has already accepted liability. However, I'm worried because I had a glass of wine at a work event before the accident. Although I don't feel I was impaired, I\n",
      "- Hi, I need to file an auto insurance claim. Last night, I was involved in a rear-end collision in California. I was driving home from work on the freeway when traffic suddenly slowed down. Unfortunately, the driver behind me didn't brake in time and hit the back of my car. The impact was significant; my car's rear bumper is severely damaged, and the trunk is crumpled. I also have some whiplash and bruising from the impact.\n",
      "\n",
      "I've been to the hospital and have the medical reports and bills ready. I also have photos of the damage to my vehicle and a copy of the police report. The other driver admitted fault and gave me their insurance information, but I'm worried because they mentioned they might have let their policy lapse. I need to claim for the damages to my car and the medical expenses. However, I'm concerned my claim might be affected because I had a glass of wine\n",
      "- Dear Insurance Agent,\n",
      "\n",
      "I hope this message finds you well. I am writing to inquire about filing an auto insurance claim following a recent accident. On October 15, 2023, at approximately 7:30 PM, I was involved in a collision in the state of California. The accident occurred on Interstate 5, near Exit 33.\n",
      "\n",
      "The incident was a rear-end collision. I was traveling at the speed limit when another vehicle hit me from behind. The cause of the accident appears to be the other driver's inattention; they were distracted by their phone. The damages to my vehicle include significant rear bumper damage, a broken taillight, and possible frame issues. Fortunately, I sustained only minor injuries—a few bruises and a sore neck. The other driver also suffered minor injuries.\n",
      "\n",
      "I would like to claim for the damages to my vehicle and any necessary medical expenses. My policy\n",
      "- Hi, I hope you're doing well. I need to file an auto insurance claim and I'm looking for some guidance on the process. Last weekend, I was involved in a rear-end collision in California. I was stopped at a red light in downtown Los Angeles when the driver behind me didn't brake in time and hit my car from the back. The impact was quite significant; my car has considerable damage to the rear bumper and trunk, and the airbags were deployed. I also sustained some injuries—mainly whiplash and a few bruises. I went to the emergency room right after the accident and have the medical reports and bills. The other driver admitted fault and we exchanged insurance information. I have their details as well as the police report from the scene.\n",
      "\n",
      "My car is a 2018 Toyota Corolla, and I'm not sure if it's drivable in its current condition. I\n",
      "--------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "# Initialize Agents\n",
    "loan_agent = LoanAgent(client, model)\n",
    "insurance_agent = InsuranceAgent(table)\n",
    "kernel_agent = KernelAgent(loan_agent, insurance_agent)\n",
    "\n",
    "# Test cases\n",
    "test_cases = [\n",
    "    {\n",
    "        \"query\": \"I need a loan for home renovation\",\n",
    "        \"params\": {\"age\": 30, \"income\": 50000, \"loan_amount\": 10000},\n",
    "    },\n",
    "    {\n",
    "        \"query\": \"I had a bad accident yesterday, but I think I was a bit drunk, need some insurance\",\n",
    "        \"params\": None,\n",
    "    },\n",
    "]\n",
    "\n",
    "# Run tests\n",
    "print(\"Testing Integrated System with Mistral:\")\n",
    "print(\"-\" * 50)\n",
    "for case in test_cases:\n",
    "    query = case[\"query\"]\n",
    "    params = case.get(\"params\")\n",
    "    result = kernel_agent.process_query(query, params)\n",
    "    print(f\"Query: '{query}'\")\n",
    "    if params:\n",
    "        print(f\"Params: {params}\")\n",
    "    if isinstance(result, dict):\n",
    "        print(f\"Decision: {result['decision']}\")\n",
    "        print(\"Similar Queries:\")\n",
    "        for q in result[\"similar_queries\"]:\n",
    "            print(f\"- {q}\")\n",
    "    else:\n",
    "        print(f\"Result: {result}\")\n",
    "    print(\"-\" * 50)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "authorship_tag": "ABX9TyOSOO1C/KFu+WsCfGtDuEGS",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
